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MEDICAL IMAGE HANDLING SYSTEM AND METHOD 

BACKGROUND OF THE INVENTION 
Field of the Invention 
5 The present invention relates to a medical image handling system and a 

medical image handling method for a medical X-ray image or the like. 
Related Background Art 

For an X-ray photographing for example for medical diagnosis, there is 
often utilized a film- screen system in which a sensitizing screen and an X-ray 

1 0 photographic film are combined. 

In such system, X-rays transmitted by an object include internal 
information of the object, and are converted by the sensitizing screen into visible 
light proportional to the intensity of the X-rays. Then the thus-converted visible 
light exposes the X-ray photographic film to form an X-ray image thereon. 

15 In a medical facility utilizing such a system, the X-ray image is 

conventionally observed by placing the film on a light box. 

On the other hand, there is recently being employed a digital X-ray image 
pickup apparatus capable of reading a radiation image as an electrical signal and 
executing a digital conversion of the read electrical signal to a digital image by 

2 0 various methods, such as a method of accumulating an X-ray intensity distribution 
as a latent image of energy in a stimulable phosphor and reading that image, a 
method of directly reading a fluorescence distribution of a fluorescent member 
formed by the X-rays, or a method of reading an image by a technology not 
utilizing the fluorescence distribution. 

2 5 Also, because digital image pickup has become possible, digital image 

diagnosis by displaying the obtained image on a display screen of a computer 
instead of outputting such image on a film, is beginning to be used. 
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On the other hand, in the medical imaging field, there has recently been 
active investigation of computer aided diagnosis (CAD), which analyzes a simple 
X-ray image or a computerized tomography (CT) image by a computer to generate 
a CAD image (diagnosis assisting image) and detecting from such CAD image 
5 (diagnosis assisting image) a portion suspected for a disease, and which is 
expected to contribute to early detection of diseases. 

SUMMARY OF THE INVENTION 

An aspect of the present invention is to judge the presence or absence of a 
1 0 diagnosis report on a medical image displayed on a monitor, and to restrict a 

change in the medical image displayed on the monitor, in a case where it is judged 

that the diagnosis report is absent. 

Another aspect of the present invention is to judge, in a case where a 

medical image displayed on a monitor is changed to a medical image of another 
1 5 subject, whether there exists another medical image different from the one 

currently displayed on the monitor but belonging to the subject of the medical 

image displayed on the monitor, and to restrict a change of subject in a case where 

there exists another, different medical image belonging to the subject of the 

medical image displayed on the monitor. 
2 0 Still other aspects of the present invention will become fully apparent 

from following description to be taken in conjunction with accompanying 

drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 
2 5 Fig. 1 is a block diagram of a medical image handling system of a first 

embodiment; 

Fig. 2 is a flow chart showing a function of the medical image handling 
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system of the first embodiment; 

Fig. 3 is a block diagram of a medical image handling system of a second 
embodiment; 

Fig. 4 is a flow chart showing a function of the medical image handling 
5 system of the second embodiment; 

Fig. 5 is a block diagram of a medical image handling system of a third 
embodiment; 

Fig. 6 is a flow chart showing a function of the medical image handling 
system of the third embodiment; 
1 0 Fig. 7 is a flow chart showing a function of a medical image handling 

system constituting a variation of the third embodiment; 

Fig. 8 is a schematic view showing a comprehensive medical system 
including a medical image handling system and constituting a fourth embodiment; 

Fig. 9 is a block diagram of a medical image handling system of a fourth 
1 5 embodiment; 

Fig. 10 is a functional block diagram showing a function of the medical 
image handling system of the fourth embodiment; 

Fig. 1 1 is comprised of Figs. 1 1 A and 1 IB showing flow charts 
illustrating a function of the medical image handling system of the fourth 
2 0 embodiment; 

Fig. 12 is a view showing a medical image display frame in the medical 
image handling system of the fourth embodiment; 

Fig. 13 is a view showing a diagnosis result input image in the medical 
image handling system of the fourth embodiment; 
2 5 Fig. 14 shows a fifth embodiment and is a block diagram showing an 

example of a configuration of a medical image handling system; 

Fig. 15 shows the fifth embodiment and is a flow chart showing a function 
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of the medical image handling system; 

Fig. 16 shows the fifth embodiment and is a view showing an example of 
a display prior to a switching of a display mode; 

Fig. 17 shows the fifth embodiment and is a view showing an example of 
5 a display after a switching of a display mode; 

Fig. 18 shows the fifth embodiment and is a view indicating an example 
of a priority of prefetching order in case of displaying a picked-up image; 

Fig. 19 shows the fifth embodiment and is a view indicating an example 
of a priority of prefetching order in case of displaying two images in laterally 
1 0 divided two image display areas; 

Fig. 20 shows a sixth embodiment and is a flow chart showing a function 
of a medical image handling system; 

Fig. 21 shows the sixth embodiment and is a view showing an example of 
display prior to a switching of an image; 
1 5 Fig. 22 shows the sixth embodiment and is a view showing an example of 

display after a switching of an image; and 

Fig. 23 shows an embodiment of the present invention and is a block 
diagram of an example of a configuration of a computer system provided in the 
medical image handling system. 

20 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

In the following, embodiments of the medical image handling system of 

the present invention will be explained with reference to Figs. 1 to 7. 
(First embodiment) 

2 5 A medical image handling system 10 of a first embodiment of the present 

invention is provided, as shown in Fig. 1, with an input part 101 such as a 
keyboard 101a and a mouse 101b, a monitor 102 for displaying a dialog image or a 
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medical image, and a control part 103 provided with a recording medium storing 
programs for controlling a communication with a medical image server (MIS) and 
a database server (DBS) and the entire medical image handling system 10. 

In the following, there will be given an explanation on how the control 
5 part 103 controls the medical image handling system 10, with reference to a flow 
chart shown in Fig. 2. 

In step S201, in response to an input from the keyboard 101a or the mouse 
101b, the control part 103 executes a control for communicating with the medical 
image server (MIS) and transferring a desired medical image for display on the 
1 0 monitor 102. 

Then step S202 starts a timer in order to count a display time of the 
medical image displayed in step S201. Usually a person who reads an image 
(hereinafter referred to as "an image reading person") starts image reading of the 
desired medical image after the medical image is displayed on the monitor 102, 
1 5 beginning a period of waiting for the end of the image reading after the timer is 
started. 

Step S203 discriminates the presence or absence of an operation for 
displaying an image other than the medical image displayed in step S201. When 
the image reading person completes the reading of the medical image displayed in 

2 0 step S201, there is executed, on the keyboard 101a or the mouse 101b, an 
operation for displaying another medical image or displaying an image for 
example of a list of patients. This step discriminates the presence or absence of 
such operation. Thus the image reading of the currently displayed medical image 
is judged to have been completed by such operation for displaying an image 

2 5 different from the currently displayed medical image. The sequence proceeds to 
step S205 in a case where step S203 identifies the presence of an operation for 
displaying an image other than the current medical image, but proceeds to step 



- 6 - 



Substitute specification (clean) 
A.N. 10/714,654 
Atty. Docket No. 03500.017720. 



S204 in the absence of such operation. 

Step S204 discriminates whether a predetermined display time has 
elapsed for the display of the medical image since the start of time counting in step 
S202. If the predetermined time has elapsed, the sequence proceeds to step S205, 
5 but, if not, the sequence returns to step S203. 

Step S205 discriminates whether an image reading result has been entered 
in an image reading result report. The sequence proceeds to step S207 or S206 
respectively in the case where an image reading result is entered or not in the 
image reading result report. In the case of an image displayed for the first time, 
1 0 the sequence always proceeds to step S206 because there is no entry of the result 
in the image reading result report. 

Step S206 displays an image for entering an image reading result report. 
The image reading person enters a result of the image reading for the medical 
image displayed in step S201, utilizing the keyboard 101a and the mouse 101b. 
1 5 The entered image reading result report is stored in the database server (DBS). 

The image for entering the image reading result report is so constructed 
that a document of a fixed pattern including an image reading result to be entered 
can be selected from a pull-down menu, in order to achieve an efficient input 
operation. 

2 0 Also, in preparation for a case that a document of a fixed pattern is 

insufficient, there is prepared an image for entering detailed information, thereby 
enabling to enter detailed information such as a name of a disease and a history 
thereof in the image reading result report. 

Also, in case of a collective inspection of a group of subjects, it is usual 

2 5 that abnormality is not found in most of the subjects. Therefore, in the absence of 
an abnormality, an input simpler than the pull-down menu is also possible by a 
single click of a button. 
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Also, it is possible to suspend the preparation of an image reading result 
report, in case the report cannot be prepared at the present stage and is to be 
prepared later by reading the image again. 

Step S207 discriminates, like step S203, the presence or absence of an 
5 operation for displaying an image other than the medical image displayed in step 
S201. Stated differently, there is awaited an operation for displaying an image 
other than the currently displayed medical image. Once such operation occurs, 
the sequence proceeds to step S208, but this step is repeated in the absence of such 
operation. 

1 0 Step S208 displays, based on the operation in step S207, an image other 

than the medical image displayed in step S201. 

As explained in the foregoing, in the case of displaying another medical 

image other than the medical image displayed in step S201, there is executed a 

control necessitating an input in the image for entering the image reading result 
1 5 report, whereby an omission of the input of the image reading result can be 

prevented. 

The input of the image reading result is not limited to a keyboard or a 
mouse, but can also be executed by another input device or by a voice input. 

(Second embodiment) 
2 0 A medical image handling system 20 of a second embodiment of the 

present invention is provided, as shown in Fig. 3, with an input part 201 such as a 
keyboard 201a and a mouse 201b, a monitor 202 for displaying a dialog image or a 
medical image, and a control part 203 provided with a recording medium storing 
programs for controlling a communication with a medical image server (MIS) and 
2 5 a database server (DBS) and the entire medical image handling system 20. 

In the following, there will be given an explanation on how the control 
part 203 controls the medical image handling system 20, with reference to a flow 
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chart shown in Fig. 4. 

In step S301, in response to an input from the keyboard 201a or the mouse 
201b, the control part 203 executes a control for communicating with the medical 
image server (MIS) and transferring a desired medical image for display on the 
5 monitor 202. 

Then step S302 starts a timer in order to count a display time of the 
medical image displayed in step S301. Usually an image reading person starts 
image reading of the desired medical image after the medical image is displayed 
on the monitor 202, beginning a period of waiting for the end of the image reading 
1 0 after the timer is started. 

Step S303 discriminates the presence or absence of an operation for 
displaying an image other than the medical image displayed in step S301. When 
the image reading person completes the reading of the medical image displayed in 
step S301, there is executed, on the keyboard 201a or the mouse 201b, an 
1 5 operation for displaying another medical image or displaying an image for 

example of a list of patients. Thus the image reading of the currently displayed 
medical image is judged to have been completed by such operation for displaying 
an image different from the currently displayed medical image. 

The sequence proceeds to step S305 in a case where step S303 identifies 
2 0 that there has been an operation for displaying an image other than the current 
medical image, but proceeds to step S304 in the absence of such operation. 

Step S304 discriminates whether a predetermined display time has 
elapsed for the display of the medical image since the start of time counting in step 
S302. If the predetermined time has elapsed, the sequence proceeds to step S305, 
2 5 but, if not, the sequence returns to step S303. 

Step S305 discriminates whether an image reading result has been entered 
in an image reading result report. The sequence proceeds to step S307 or S306 
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respectively according as an image reading result is entered or not in the image 
reading result report. In a case of an image being displayed for the first time, the 
sequence always proceeds to step S306 because there is no entry of the result in 
the image reading result report. 
5 Step S306 automatically generates an image reading result report. A 

content of the automatically generated image reading result report can be set in 
advance. Since there is often prepared a report indicating the absence of any 
abnormal observation, there is set "no observation". 

In this operation, there are automatically entered a display time of the 
1 0 image as a time required for image reading, and a name of the image reading 

person in the image reading result report. Upon generation of the image reading 
result report, the sequence proceeds to step S307. 

Step S307 stores the image reading result report, generated in step S306, 
in the database server DBS. After the storage of the image reading result report, 
1 5 the sequence proceeds to step S308. 

Step S307 discriminates, like step S303, the presence or absence of an 
operation for displaying an image other than the medical image displayed in step 
S201. Stated differently, there is awaited an operation for displaying an image 
other than the currently displayed medical image. In the presence of such 
2 0 operation, there is displayed an image other than the currently displayed medical 
image, but this step is repeated in the absence of such operation. 

As explained above, the present embodiment displays a medical image, 
and generates an image reading result report automatically in case the image 
reading result report is not entered after the lapse of a predetermined time. 
2 5 The input of the image reading result is not limited to a keyboard or a 

mouse, but can also be executed by another input device or by a voice input. 

(Third embodiment) 
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A medical image handling system 30 of a third embodiment of the present 
invention is provided, as shown in Fig. 5, with an input part 301 such as a 
keyboard 301a and a mouse 301b, a monitor 302 for displaying a dialog image or a 
medical image, and a control part 303 provided with a recording medium storing 
5 programs for controlling a communication with a medical image server (MIS) and 
a database server (DBS) and the entire medical image handling system 30. 

In the following, there will be given an explanation on how the control 
part 303 controls the medical image handling system 30, with reference to a flow 
chart shown in Fig. 6. 

10 In step S401, in response to an input from the keyboard 301a or the mouse 

301b, the control part 303 executes a control for communicating with the medical 
image server (MIS) and transferring a desired medical image for display on the 
monitor 302. 

Step S402 discriminates presence or absence of an operation for 
1 5 displaying an image other than the medical image displayed in step S401. When 
the image reading person completes the reading of the medical image displayed in 
step S401, there is executed, on the keyboard 301a or the mouse 301b, an 
operation for displaying another medical image or displaying an image for 
example of a list of patients. Thus the image reading of the currently displayed 
2 0 medical image is judged to have been completed by such operation for displaying 
an image different from the currently displayed medical image. 

Step S403 discriminates whether an image reading result has been entered 
in an image reading result report. The sequence proceeds to step S406 or S404 
respectively according as an image reading result is entered or not in the image 
2 5 reading result report. In a case of an image being displayed for the first time, the 
sequence always proceeds to step S404 because there is no entry of the result in 
the image reading result report. 
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Step S404 automatically generates an image reading result report. A 
method of automatic generation of the image reading result report is similar to that 
in the second embodiment and will not be explained further. Upon generation of 
the image reading result report, the sequence proceeds to step S405. 
5 Step S405 stores the image reading result report, generated in step S404, 

in the database server DBS. After the storage of the image reading result report, 
the sequence proceeds to step S406. 

Step S406 executes an image display corresponding to the operation in 
step S402. For example, if step S402 executes an operation for displaying a 
1 0 medical image of a next subject, step S406 displays the medical image of the next 
subject. Also, if step S402 executes an operation of displaying a list of medical 
images, step S406 displays the list of the medical images. 

As explained above, the present embodiment displays a medical image, 
then, in the case of an operation for displaying an image other than the displayed 
1 5 medical image, there is discriminated presence or absence of an input of an image 
reading result report, generates an image reading result report automatically in case 
the image reading result report is not entered. 

The input of the image reading result is not limited to a keyboard or a 
mouse, but can also be executed by another input device or by a voice input. 
2 0 (Variation of third embodiment) 

Fig. 7 is a flow chart showing a case in the foregoing third embodiment 
where a subject waits for plural inspections. 

An inspection is defined by the DICOM standard and is from the start of 
an image taking operation to the end thereof. For example, in case of taking a 
2 5 chest X-ray image, an image of the chest part is taken from the start of an image 
taking operation, and the image taking operation is thus terminated. In this case, 
there will be provided a chest X-ray image for an inspection. 
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In step S501, in response to an input from the keyboard 301a or the mouse 
301b, the control part 303 executes a control for communicating with the medical 
image server (MIS) and transferring a desired medical image for display on the 
monitor 302. 

5 Step S502 discriminates presence or absence of an operation for 

displaying an image other than the medical image displayed in step S501. When 
the image reading person completes the reading of the medical image displayed in 
step S501, there is executed, on the keyboard 301a or the mouse 301b, an 
operation for displaying another medical image or displaying an image for 
1 0 example of a list of patients. Thus the image reading of the currently displayed 
medical image is judged to have been completed by such operation for displaying 
an image different from the currently displayed medical image. 

Step S503 discriminates whether an image reading result has been entered 
in an image reading result report. The sequence proceeds to step S506 or S504 
1 5 respectively according as an image reading result is entered or not in the image 

reading result report. In a case of an image being displayed for the first time, the 
sequence always proceeds to step S504 because there is no entry of the result in 
the image reading result report. 

Step S504 automatically generates an image reading result report. A 
2 0 method of automatic generation of the image reading result report is similar to that 
in the second embodiment and will not be explained further. Upon generation of 
the image reading result report, the sequence proceeds to step S505. 

Step S505 stores the image reading result report, generated in step S504, 
in the database server DBS. After the storage of the image reading result report, 
2 5 the sequence proceeds to step S506. 

Step S506 discriminates whether there is a next inspection for the 
currently displayed subject. Since the monitor displays an image of a latest 
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inspection of the subject, if step S501 instructs the display of an image taken at 
10:10 a.m., April 1, 2002, the "next inspection" mentioned above means an 
inspection taken before 10:10 a.m. April, 1, 2002. 

In a case where there is an inspection taken before the date of the medical 
5 image displayed in step S501, for example an inspection taken at 1 1:40 a.m., 

March 1, 2001, the sequence proceeds to step S508, but, in a case where there is no 
inspection taken before the date of the medical image displayed in step S501, the 
sequence proceeds to step S507. 

Step S508 displays a medical image of an inspection taken before the date 
10 of the medical image displayed in step S501, and the sequence proceeds to step 

S503. For example, a medical image of an inspection taken at 1 1:40 a.m., March 
1, 2001, is displayed and the sequence proceeds again to step S503. Thereafter, 
the sequence proceeds to step S507 when the next sequences are exhausted. 

Step S507 executes an image display corresponding to an operation 
1 5 content in step S502. 

(Fourth embodiment) 

Fig. 8 is a view showing an entire comprehensive medical system 
including a medical image handling system and constituting a fourth embodiment, 
Fig. 9 is a block diagram showing an internal configuration of the medical image 

2 0 handling system, and Fig. 10 is a functional block diagram showing a function of 
the medical image handling system. 

Referring to Fig. 8, a medical image handling system 110 is connected, 
through a local area network (LAN) 100, to medical image generating apparatuses 
130, 140, 150 and a medical data server 160. The medical image generating 

2 5 apparatuses 130, 140 are for example a CT scan apparatus and an MRI apparatus, 
and the medical image generating apparatus 150 is for example an X-ray image 
pickup apparatus. 
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Medical images generated in the medical image generating apparatuses 
130, 140, 150 are transmitted, either directly or through the medical data server 
160, to the medical image handling system 1 10. The medical image transmitted 
directly to the medical image handling system 1 10 is stored in a memory medium 
5 thereof. 

The medical image handling system 1 10 is provided with a general- 
purpose color liquid crystal monitor 120, thereby being capable of easily 
displaying a medical image. There can also be utilized a medical image handling 
system 180 provided with a monochromatic medical image monitor 170 of a 

1 0 higher definition. 

In response to an instruction of an image reading person, the medical 
image handling system 110 displays the received medical image, and the image 
reading person can read the medical image and enter a result of such image reading 
in the medical image handling system 110. 

1 5 Referring to Fig. 9, the medical image handling system 1 10 is formed by 

connecting, to a bus 200, a CPU 210, a RAM 220, a ROM 230, a communication 
interface 240 and input means 260, and output apparatuses such as general-purpose 
color liquid crystal monitor 120, a printer 250, etc., are connected to the bus 200 
through an appropriate interface. The input means includes a keyboard, a 

2 0 pointing device, a microphone etc. 

The CPU 210 is used for controlling the entire medical image handling 
system 110 and the output apparatuses, and a control program therefor is stored in 
the ROM 230. A communication interface 240 controls a communication by the 
LAN 100, and exchanges medical image and other data with the medical image 

2 5 generating apparatuses 130, 140, 150 and the medical data server 160. 

Referring to Fig. 10, the medical image handling system 110, upon 
receiving medical data Md including a medical image, displays the medical image 
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and other medical data by a medical image handling system 300 (corresponding to 
the general-purpose color liquid crystal monitor 120 in Fig. 8 and the CPU 210 and 
the RAM 220 in Fig. 9). The image reading person reads the medical image and 
enters and stores a result of image reading by first input storage means 310 
5 (corresponding to the medical data server 160 in Fig. 8 and the CPU 210, the RAM 
220, the communication interface 240 and the input means 260 in Fig. 9). 

The first input storage means 310 is capable of a process of suspending 
the input of the result of image reading, and, in such case, the medical data record 
that the result of image reading is not recorded but suspended. In this manner the 

1 0 image reading person can enter the result of image reading at an arbitrary timing 
and can avoid a lapse in recording. 

Such suspension is effective for example in a case of browsing an entire 
collection of images as in a slide show, in a case of screening by an image reading, 
in a case where a judgment is not easy, in a case where a second reading later is 

1 5 desirable, or in a case where an input is intentionally not desired. 

The medical data once stored can be displayed again by image display 
designation means 320 (corresponding to the input means 260, CPU 210 and RAM 
220 in Fig. 9), and the image reading person can confirm the medical data by 
confirmation means 330 (corresponding to the general-purpose color liquid crystal 

2 0 monitor 120 in Fig. 8 and the input means 260, CPU 210 and RAM 220 in Fig. 9). 
For the redisplayed medical image, a result of image reading can be entered or 
edited by second input storage means (corresponding to the medical data server 
160 in Fig. 8 and the CPU 210, RAM 220, communication interface 240 and input 
means 260 in Fig. 9). The medical image after such input and editing is 

2 5 displayed, under a condition designated by the image display designation means 
320, in image display means 350 (corresponding to the general-purpose color 
liquid crystal monitor 120 in Fig. 8 and the CPU 210 and the RAM 220 in Fig. 9). 
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Presence of such second input storage means allows the image reading 
person to record the result of image reading at an arbitrary timing. 

It is also possible to limit the medical image that can be redisplayed, to a 
medical image for which the result of image reading is suspended. In this manner 
5 such suspended medical image can be automatically extracted, thereby avoiding a 
lapse in the record. 

In the following there will be explained a medical image reading method 
to be executed in the medical image handling system 110. 

Figs. 1 1 A and 1 IB are flow charts showing a process in the fourth 
1 0 embodiment of the medical image processing method, Fig. 12 is an elevation view 
showing a medical image display frame in the fourth embodiment, Fig. 13 is an 
elevation view showing an image reading result entering image in the fourth 
embodiment, and Fig. 14 is a conceptual view showing a data structure of the 
medical data in the fourth embodiment. 
15 A medical image reading method shown in a flow chart in Figs. 1 1 A and 

1 IB are executed in the following manner: 

Step S401: It is assumed that an image is displayed for each subject, and a 
subject number "i" corresponding to the medical image and an interval (step) "s" 
of the displayed medical images are set at a default value "1". In this manner the 
2 0 medical images are displayed in succession from a first image. Then the 
sequence proceeds to step S402. 

Step S402: An i-th medical image is displayed on the general-purpose color 
liquid crystal monitor 120. 

As shown in Fig. 12, the general-purpose color liquid crystal monitor 120 
2 5 displays a medical image display area 500, a subject data display area 510, an 
instruction 520 for image designation, instructions 530, 540, 550 for a display 
mode change, and a step designation area 560. The subject data display area 510 
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displays a patient ID, patient name, sex and birth date of a subject corresponding to 
the medical image. These are based on the DICOM standard. 
Then the sequence proceeds to step S403: 
Step S403: a timer is started in order to count an image reading time. Then 
5 the sequence proceeds to step S404: 

Step S404: There is discriminated whether the image reading person has 
instructed to interrupt the display of the currently displayed medical image and to 
display a next medical image. The instruction 520 for image designation is 
constituted of buttons of "FORWARD", "REVERSE" and "LIST", and a 
1 0 depression of the "FORWARD" button designates a next medical image. The 

sequence proceeds to step S408 or S405 respectively according as the display of a 
next medical image is instructed or not. 

Step S405: There is discriminated whether the image reading person has 
instructed to interrupt the display of the currently displayed medical image and to 
1 5 display a preceding medical image. A depression of the "REVERSE" button in 
the instruction 520 for image designation designates a preceding medical image. 
The sequence proceeds to step S409 or S406 respectively according as the display 
of a previous medical image is instructed or not. 

Step S406: There is discriminated whether the image reading person has 
2 0 instructed to interrupt the display of the currently displayed medical image and to 
display a list of subjects. A depression of the "LIST" button in the instruction 
520 for image designation designates a display of a list of the subjects. The 
sequence proceeds to step S407 or S41 1 respectively according as the display of a 
list of the subjects is instructed or not. 
2 5 Step S407: A list of persons who take inspections 

(subjects) is displayed, and the sequence proceeds to step S410. 

Step S408: In case a display of a next medical image is instructed despite that 
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the current medical image is the last medical image, there is displayed a message 
of such effect and the sequence returns to step S402. 

Step S409: In case a display of a preceding medical image is instructed despite 
that the current medical image is the first medical image, there is displayed a 
5 message of such effect and the sequence returns to step S402. 

Step S410: There is discriminated whether the image reading person has 
designated a subject in the list of the subject displayed in step S407. In a case 
where a subject is designated, a medical image number i thereof is acquired and 
the sequence returns to step S402. If the subject is not designated, there is 

1 0 discriminated whether a change in the display mode for the currently displayed 
medical image. In the image display mode, there can be executed an image 
scaling-up, a scaling-down, a contrast adjustment and a distance measurement, and 
there can also be set a step s. The sequence proceeds to step S412 or S413 
respectively according as a change in the display mode is instructed or not. 

1 5 Step S41 1: a change in the display mode is executed. An instruction for the 

change of the display mode is constituted of an image scale-up/down button 530, a 
contrast adjustment button 540 and a distance measurement button 550. In the 
image scale-up/down button 530, a depression of a rightward arrow button or a 
leftward arrow button respectively causes an enlargement or a reduction of the 

2 0 image. Also, in the contrast adjustment button 540, a depression of a rightward 
arrow button or a leftward arrow button respectively causes an increase or a 
decrease in the contrast. A depression of the distance measurement instruction 
button 550 executes a distance measurement. In a step designation area, there are 
provided a window 542 for displaying a current step and increase/decrease arrows 

2 5 544, and a depression of an upward arrow increases the step and an increased step 
is displayed in the window 542. Also a depression of a downward arrow 
decreases the step and a decreased step is displayed in the window 542. 
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Then the sequence proceeds to step S412: 
Step S412: There is executed a change in the display mode designated in step 
S411; 

Step S413: There is discriminated whether an image reading time, of which 
5 counting was started in step S403 has expired. The sequence proceeds to step 
S414 or returns to step S404 respectively in case the image reading time has 
expired or not; 

Step S414: There is discriminated, on the currently displayed medical image, 
whether a result of image reading has been inputted (not suspended). The 
1 0 sequence proceeds to step S421 or S415 respectively in case the result of image 
reading has been inputted (not suspended) or not; 

Step S415: An input image for inputting a result of image reading is displayed. 
By automatically displaying an input image at the expiration of the image reading 
time in this manner, it is possible to alleviate the burden on the image reading 
1 5 person and to prevent a lapse in the input. By providing a certain voice message 
simultaneously with the display of the input image, it is possible to enhance the 
effect of preventing a lapse in the input. 

As shown in Fig. 13, the input image includes a medical image display 
area 600, a subject data display area 610, a detailed information input area 620, a 
2 0 sound input designation button 630, a hold selection button 640 and a standard 
document selection area 650. 

Then the sequence proceeds to step S416: 
Step S416: A timer for measuring an image reading result input waiting time is 
started; 

2 5 Step S417: There is discriminated whether the image reading person has 

selected a suspension of the image reading result. The suspension is selected by 
depressing the hold button 640. The sequence proceeds to step S421 or S418 
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respectively in case the suspension is selected or not; 

Step S418: There is discriminated whether the image reading person has 
inputted a result of image reading. The image reading result can be inputted by 
entering a text in the detailed information input area 620, or by executing a sound 
5 input under a depression of the sound input designation button 630, or by selecting 
a standard document in the standard document selecting area 650. A standard 
document can be selected by various methods, for example by selecting a standard 
document from a pull-down menu, or by suitably scrolling standard documents 
displayed in the display area 652 by a scroll button 654. 

1 0 The sequence proceeds to step S421 or S419 respectively in case the 

result of image reading is inputted or not: 

Step S419: There is discriminated whether a waiting time for input of the result 
of image reading has expired. The sequence proceeds to step S420 or returns to 
step S417 respectively according as the waiting time for input of the result of 

1 5 image reading has expired or not; 

Step S420: A state in which the waiting time for input of the result of image 
reading has expired without the input is regarded as a suspension and the sequence 
proceeds to step S421. In this manner there can be prevented a lapse in the input; 
Step S421: There is designated a medical image advanced by the internal s of 

2 0 the medical images, whereby the images are advanced automatically to alleviate 
the burden of the image reading person. Particularly in case of browsing all the 
images without entering the result of image reading, the image reading person can 
execute such browsing without any operation, whereby the burden of the image 
reading person can be significantly reduced; 

2 5 Step S422: There is discriminated whether the image reading of the medical 

image has been completed, and the sequence is terminated or returns to step S402 
respectively according as the image reading is completed or not. 
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The fourth embodiment provides a suspension process, thereby enabling 
the image reading person to input the result of image reading at an arbitrary timing 
and to prevent a lapse in the input. 

(Fifth embodiment) 

5 In the following, a fifth embodiment of the present invention will be 

explained in detail with reference to accompanying drawings. 

Recently CAD of various types are conceived depending on an image or 
an object to be considered, but a CAD for obtaining a difference of images of a 
same site thereby forming an image emphasized in a change in time is attracting 
1 0 attention. 

In a medical facility utilizing such CAD, it has been common to compare 
two images different in time on a light box or the like, in order to judge a 
proceeding of a disease or a progress in a treatment. 

However, since the digital image pickup has become possible as 

1 5 explained in the foregoing, it is conceived possible, for example, to execute an 
image analysis on a set of simple chest X-ray images taken at different times 
thereby determining an anatomically same position in each image, and to deform 
either one of the present image or the past image thereby executing a differential 
process on each pixel. 

2 0 A density value of the image obtained by such differential process 

(differential process) corresponds to a difference in the image signal between the 
present and past images. More specifically, the differential between these images 
becomes zero in case the present image and the past image show no change, but, in 
case there is any variation between the present image and the past image, the 

2 5 luminance level shows a change corresponding to such variation. 

A judgment with such differential image can clearly and positively 
indicate a progress in the treatment or a proceeding in the disease in comparison 
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with a judgment of such progress of the treatment or the proceeding of the disease 
based on a mere comparison of the past and present images in human brain, and a 
display of such differential image is very effective means for preventing an 
erroneous diagnosis or an error in the judgment of a doctor. 
5 For example, in case there are three or more images different in time, 

there can be generated plural differential images, thereby allowing to more exactly 
know a change in the image over a prolonged period. 

Fig. 14 is a block diagram showing a configuration of a medical image 
handling system of the present embodiment. 

1 0 Referring to Fig. 14, there are provided image reading means 1101, image 

processing means 1 102, image display means 1 103, divided display format setting 
means 1104, display mode setting means 1105, image switching means 1106, 
taken images 1 107, diagnosis supporting images 1 108, and a monitor 1 109. In 
the present embodiment, it is assumed that the taken images 1 107 and the 

1 5 diagnosis supporting images 1 108 are already stored in the memory media. 

In the following, there will be explained a function and an operation of the 
medical image handling system of the present embodiment, constructed as 
explained above. Fig. 15 is a flow chart for switching a display of a taken image 
and a diagnosis supporting image, based on an instruction for changing a display 

2 0 mode setting. 

At first in step SI 201, the image reading means 1101 reads a set value of 
a divided display image format. The divided display image format includes, for 
example, a single image display, a left-right two-image display, an upper-lower 
two-image display, a four-image display and a display of more images, and is 

2 5 selected according to an operator operation utilizing the divided display format 
setting means 1104. 

The images displayed in the respective image display areas are those of a 
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same subject, distanced in time. The medical image is usually displayed 
according to a DICOM (digital and communication medicine) standard. 

According to the DICOM standard, each of the images distant in time is 
called a study. Therefore each patient has plural studies different in time. 
5 Then, in step S1202, the image reading means 1101 reads a display mode 

for each study. The display mode is set based on an operation of the operator 
utilizing the display mode setting means 1 105, and, in the present embodiment, 
includes a mode of displaying a taken image and a mode of displaying a diagnosis 
supporting image. 

1 0 The diagnosis supporting image is for example a time-differential image, 

but such example is not restrictive and there may be employed any image capable 
of supporting the diagnosis of the image reading doctor. 

Then, in step S1203, the image reading means 1101 reads image data. 
More specifically, the image reading means 1101 reads necessary image data 

1 5 according to set values, set in steps S1201 and S1202. 

For example, in a case of a divided display format for displaying taken 
images both at right and left, there are read, among the studies of the patient, 
image data of a most recent study and those of a study preceding to the most recent 
study. 

2 0 Also, in a case where the divided display format is for displaying two left- 

right images and a display mode for the left image display area is a mode for 
displaying a taken image and a display mode for the right image display area is a 
mode for displaying a diagnosis supporting image, there are read, among the 
studies of the patient, a most recent study and an image of a difference between the 

2 5 most recent study and a study preceding the most recent study, namely a time- 
differential image between the present image and the immediately preceding image. 
Then, in step S1204, the image processing means 1102 executes an image 
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processing on the image data read in step S1203. Usually it is possible, by 
selecting the divided display format, to acquire and determine a size of each image 
display area, a gamma value of the monitor, a lookup table for display, and a 
displayed image size. 

5 Then, based on these parameters, an image processing is executed on the 

image data read in step S1203. However, the parameters acquired and 
determined in step SI 204 are not limited to those mentioned above. 

Then, in step S1205, the image display means 1103 displays, on a monitor 
1 109, an image subjected to the image processing in step S1204. The image 

1 0 displayed on the monitor 1 109 is read usually by an image reading doctor. 

Then, in step S1206, the image reading means 1101 checks whether an 
instruction for changing the aforementioned display mode has been given from the 
operator. Such instruction for change is given by the operator, utilizing the image 
switching means 1106. 

15 In a case where a change in the display mode is instructed, the image data 

to be displayed are determined according to such instruction. For this purpose, 
the S1203 reads the determined image data to be displayed, then step S1204 
executes an image processing on the read image data, and step S1205 displays the 
image, subjected to such image processing, on the monitor 1109. 

2 0 The instruction for changing the display mode is executed, for example as 

shown in Fig. 16, by displaying a button 1303 for switching the display mode 
together with the images 1301, 1302 in the process of step S1205 and by 
depressing such button 1303 by an operator of the operator. 

In Fig. 16, a symbol 1303 indicates a button for switching the display 

2 5 mode. However, operation means for switching the display mode is not limited 
to such button. 

Also in Fig. 16, a symbol 1301 indicates a latest image of the patient, 
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which in this example is a front chest image taken on March 27, 2002. Also a 
symbol 1302 in Fig. 16 indicates an image which immediately precedes the latest 
image among images of a same site of the same patient, and which in this example 
is an image taken on September 9, 2002. In this manner it is possible to compare 
5 the image taken at the present time and the image taken previously. 

By clicking the button 1303 with a mouse, the display mode can be 
changed from a mode of displaying a taken image to a mode of displaying a time- 
differential image which is a diagnosis supporting image. 

When the button 1303 is clicked in the state shown in Fig. 16 to select the 

1 0 time-differential image display mode, a time-differential image is prepared from 
the latest image 1301 displayed in the left image display area and the image 1302 
displayed in the right image display area. Then, as shown in Fig. 17, the latest 
image 1301 on the monitor 1 109 is retained in the displayed state, and a time- 
differential image 1401, prepared from the latest image 1301 and the image 1302 

1 5 immediately preceding the latest image, is displayed at a side of the latest image 
1301. 

In this manner the display mode is switched from the mode for displaying 
the taken image to the mode for displaying the diagnosis supporting image. 

In the present embodiment, there is explained a case of executing the 
2 0 process by reading the images in succession, but, in the actual image reading site, 
it is preferable to pre-fetch the images since the image reading takes time. 

Such pre-fetch is possible also in the present embodiment, but it is 
necessary to vary a priority of images to be pre-fetched according to the divided 
display format or the display mode. 
2 5 Fig. 18 shows an example of the pre-fetching priority in a display mode of 

displaying the taken images. 

In an example shown in Fig. 18, in case of a single-image (No division) 
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display, into which a display area is divided, a priority is given in an order of a 
first series, a second series and a third series in a first study (study 1). 

Also, in a case of a two-image (2-division) display, a priority is given in 
an order of a first series in an initial study (study 1), a first series in a second study 
5 (study 2), a second series in the initial study (study 1), and a third series in the 
initial study (study 1). 

Also, in a case of a four-image (4-division) display, a priority is given in 
an order of a first series in an initial study (study 1), a first series in a second study 
(study 2), a first series in a third study (study 3), a first series in a fourth study 
1 0 (study 4), a second series in the initial study (study 1), and a third series in the 
initial study (study 1). 

Also, Fig. 19 shows an example of the pre-fetching priority in a left-right 
two-division format. 

In a case of displaying the taken images in left and right image portions of 
1 5 the monitor 1 109, a priority is given, as shown in Fig. 19, in an order of a taken 
image in a first study (study 1), a taken image in a second study (study 2), a 
differential image of the taken image in the first study and the taken image in the 
second study (CAD study 1 - study 2), a taken image in a third study (study 3), a 
taken image in a fourth study (study 4), a differential image of the taken image in 
2 0 the second study and the taken image in the third study (CAD study 2 - study 3), 
and a differential image of the taken image in the third study and the taken image 
in the fourth study (CAD study 3 - study 4). 

Also, in a case of displaying a taken image in a left half image portion of 
the monitor 1 109, and a diagnosis supporting image in a right half image portion, a 
2 5 priority is given in an order of a taken image in a first study (study 1), a differential 
image of the taken image in the first study and the taken image in the second study 
(CAD study 1 - study 2), a taken image in a second study (study 2), a taken image 
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in a third study (study 3), a taken image in a fourth study (study 4), a differential 
image of the taken image in the second study and the taken image in the third 
study (CAD study 2 - study 3), and a differential image of the taken image in the 
third study and the taken image in the fourth study (CAD study 3 - study 4). 
5 As explained in the foregoing, the present embodiment sets at least an 

image display area in a divided display image format and also sets either of the 
taken image 1107 and the diagnosis supporting image (differential image) 1108 in 
thus set image display area thereby displaying the set image in the set image 
display area, whereby the taken image 1 107 or the differential image 1 108 can be 

1 0 displayed in an optimum form and the differential image 1 108 can be referred to at 
the image reading within a same image frame displaying the taken image 1 107. 

Also, since there are set plural image display areas and the taken image 
1 107 and the diagnosis supporting image (differential image) 1 108 are displayed in 
thus set plural image display areas, it is rendered possible to positively indicate the 

1 5 differential image 1 108 simultaneous with a diseased part of the original taken 

image 1 107, thereby allowing to promptly and securely understand the proceeding 
of the diseased part or the progress of the treatment between studies distanced in 
time, in comparison with the prior method in which the diseased part is understood 
by the doctor through a comparison of the taken images. 

2 0 Also, the present embodiment, employing a CAD image as the diagnosis 

supporting image, allows to employ and display images of various forms as the 
diagnosis supporting image. In this manner, the image reading doctor can display 
a desired image and can derive a result of image reading by arbitrarily referring to 
various images. 

2 5 Also, when the image reading doctor reads an image on a monitor or a 

film, a size or a shape of a diseased part may be unconsciously deformed in case a 
sight line is displaced from a point to be noticed. For this reason, the image 
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reading doctor generally dislikes a large displacement of the sight line in the 
course of the image reading. 

For example, in case of displaying images on two monitors and 
comparing the left and right images displayed on such two monitors, it is necessary, 
5 after reading the image displayed on a monitor, to move the slight line 

significantly to the other monitor. This may also interrupt the consideration of 
the doctor. Thus, in case of comparing two images, a displacement of the sight 
line from the point to be noticed may result in a deterioration of the image reading 
efficiency. 

1 0 On the other hand, in the present embodiment, since the taken image and 

the diagnosis supporting image are displayed in a same (alternate) image display 
area, the image reading doctor can refer to the taken image and the diagnosis 
supporting image without a large displacement of the sight line. In this manner it 
is possible to improve the image reading efficiency. Particularly in case of 

1 5 comparing the size or the shape of the diseased part, the image reading doctor can 
make such comparison without interrupting the consideration, whereby a very 
significant effect can be obtained. 

Also, in a case where the images are not pre-fetched, when the image 
reading doctor instructs the display of a desired image, there are executed a 

2 0 loading of the image and an optimum image processing before the image is 
displayed on the monitor, whereby the display has a poor response and the 
efficiency of diagnosis is significantly deteriorated. 

On the other hand, in the present embodiment, since a priority of the 
images to be pre-fetched is determined according to the divided display format and 

2 5 the display mode (pre-fetching condition being switched), the images are read into 
the memory always in an optimum condition. Consequently, the image reading 
doctor can display a desired image on the monitor 1 109 at a desired time, without 
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interruption of the consideration, thereby an improvement is achieved in the 
efficiency of image diagnosis and also in the precision of diagnosis. 
(Sixth embodiment) 

In the following, a sixth embodiment of the present embodiment will be 
5 explained in detail. This embodiment is different from the fifth embodiment in a 
process in case an instruction for switching the image is given. Therefore 
portions that are the same as those in the foregoing fifth embodiment are indicated 
by symbols same as those in Figs. 14 to 19 and detailed explanations on these 
portions will be omitted. 
10 In the following, there will be explained a function and an operation of the 

medical image handling system of the present embodiment, with reference to Fig. 
20. Fig. 20 is a flow chart for, in a case of an instruction for changing a taken 
image from the operator, switching the diagnosis supporting image in linkage. 
Steps S1701 to S1705 are the same as steps S1201 to S1205 of the 
1 5 foregoing fifth embodiment explained in Fig. 15, and will not, therefore, be 
explained further. 

In step S1706, the image reading means 1101 discriminates whether an 
image switching is instructed from the operator. The sequence proceeds to step 
S1707 or is terminated respectively in case the instruction for the image switching 
2 0 is present or absent. 

In the following a specific operation for the image switching will be 
explained. As an example, Fig. 21 shows a case where the divided image format 
is a left-right two division, in which the left image display area is set at a mode of 
displaying a taken image and the right image display area is set at a mode of 
2 5 displaying a diagnosis supporting image. 

In Fig. 21, a symbol 1803 indicates an image switching button. An 
image switching instruction is generated by clicking of the image switching button 
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In step S1707, the image reading means 1101 discriminates whether a link 
inhibition flag is turned on or off. The sequence is terminated in case the link 
inhibition flag is on, but, in case the link inhibition flag is off, the sequence returns 
5 to step S1703 to read a next image and step S1704 to execute the image processing, 
and step S1705 displays a next image. 

In the following, there will be explained a specific operation of the link 
inhibition flag. In Fig. 21, the link inhibition flag is turned on/off by a depression 
of a link inhibition flag setting button 1804 by the operator. However a method 
1 0 for on/off switching of the link inhibition flag is not limited to such example. 

In Fig. 21, the link inhibition flag is turned off. 

Therefore, the diagnosis supporting image is switched in linkage with an 
image switching instruction of the operator in step S1706. 

On the other hand, in a case where the link inhibition flag is turned on, 
1 5 when an image switching instruction is given in step S1706, the diagnosis 
supporting image is prevented from switching and retains a state before the 
switching instruction. 

Such image switching operation will be explained with reference to Fig. 
21. Fig. 21 shows a state before the image switching instruction is given. In Fig. 
2 0 21, a taken image 2801, which is a latest study of the patient, taken on March 27, 
2002, is displayed in the left image display area. 

Also, the right image display area displays a time-differential image as a 
diagnosis supporting image. In this example, there is displayed a time- 
differential image prepared from an image taken in a study on March 27, 2002 and 
2 5 an image taken in a study on September 9, 2001. 

In a case where the operator clicks the image switching button 1803 with 
the mouse while these images are displays, the two images displayed in the left 
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and right image display areas are switched in linkage. Fig. 22 shows a state after 
the left and right two images are switched in linkage by the clicking of the image 
switching button 1803. The taken image 1801 of the latest study, displayed in the 
left image display area in Fig. 21, is switched to a taken image 1901 taken in a 
5 study of September 9, 2001, immediately preceding the latest study. 

Also, the diagnosis supporting image displayed in the right image display 
area in Fig. 22 is switched in linkage with the taken image displayed in the left 
image display area. 

Stated differently, in the state shown in Fig. 21, the left image display area 

1 0 displays the taken image 1801 obtained in the study of March 27, 2002, and the 

right image display area displays the time-differential image of the image of March 
27, 2002, and the image of September 9, 2001. In Fig. 22 after the switching, the 
left image display area displays the taken image 1901 obtained in the study of 
September 9, 2001, and the right image display area displays the time-differential 

1 5 image of the image of September 9, 2001, and the image of September 25, 2000. 
The image of September 25, 2000, is an image taken in a study preceding the 
image of September 9, 2001. Therefore, this image was taken in a second study 
before the latest image. 

Thus, this patient was subjected to front chest X-ray imaging on March 27, 

2 0 2002, September 9, 2001, and September 25, 2000. Therefore, at the observation 
of the image of March 27, 2002, there is simultaneously displayed a time- 
differential image prepared from the image of September 9, 2001, which is 
immediately preceding (older than) the image of March 27, 2002, and the image of 
March 27, 2002, and, at the observation of the image of September 9, 2001, there 

2 5 is simultaneously displayed a time-differential image prepared from the image of 
September 25, 2000, which is immediately preceding (older than) the image of 
September 9, 2001, and the image of September 9, 2001. 
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In the present embodiment, as explained in the foregoing, in a state where 
the taken image 1801 in the latest study is displayed in the left image display area 
and the time-differential image 1802, prepared from the taken image 1801 in the 
latest study and the taken image in the immediately preceding study, is displayed 
5 in the right image display area, when the taken image 1801 is switched to the taken 
image 1901 taken in an immediately preceding study, the time-differential image 
displayed in the right image display area is also switched in linkage to a time- 
differential image 1902 prepared from the taken image in a study immediately 
preceding the latest study and the taken image in a study second preceding the 
1 0 latest study. 

Thus, since, at the switching of the taken image, the diagnosis supporting 
image is switched in linkage with such switching of the taken image, the taken 
image and the diagnosis supporting image can always be displayed as a pair of 
images. The relationship between the taken image and the diagnosis supporting 
1 5 image can be understood more easily. 

For example, in a case where each of the taken image and the time- 
differential image (diagnosis supporting image) is present in plural units, such 
time-differential images are very similar, so that it is very difficult, in a mere 
display of a time-differential image, to immediately identify such time-differential 
2 0 image is obtained from which ones of the taken images. 

Also, it has not been possible to switch the display of a first taken image, 
a second taken image and a third taken image, a differential image of the first 
taken image and the second taken image, and a differential image of the second 
taken image and the third taken image, so that a correspondence between a taken 
2 5 image and a time-differential image (diagnosis supporting image) has been 
difficult to understand. 

In contrast, the present embodiment switches the time-differential image 



- 33 - 



Substitute specification (clean) 
A.N. 10/714,654 
Atty. Docket No. 03500.017720. 



(diagnosis supporting image) in linkage with a switching operation of the taken 
image, thereby enabling to display the taken image and the diagnosis supporting 
image in a pair and to immediately understand the nature of the time-differential 
image (diagnosis supporting image). 
5 The medical image handling system explained in the foregoing 

embodiments can be realized by a computer system as shown in Fig. 23. 

Fig. 23 is a block diagram showing an example of a computer system 
provided in the medical image handling system. 

In Fig. 23, a computer system 1050 is constituted of a CPU 1051, a ROM 
1 0 1052, a RAM 1053, a keyboard controller (KBC) 1055 for a keyboard (KB) 1054, 
a CRT controller (CRTC) 1057 for a CRT display (CRT) 1056 constituting a 
display unit, a disk controller (DKC) 1060 for a hard disk (HD) 1058 and a flexible 
disk (FD) 1059, and a network interface controller (NIC) 1062 for a network 1061, 
connected through a system bus 1063 in mutually communicable manner. 
1 5 The CPU 1051 collectively controls the components connected to the 

system bus 1053 by executing a software stored in the ROM 1052 or the HD 1058, 
or a software supplied from the FD 1029. 

Thus the CPU 1051 executes control for realizing following operations, 
by reading and executing a program of predetermined process sequence from the 
2 0 ROM 1052, the HD 1058, or the FD 1029. 

The RAM 1053 functions as a main memory or a work area of the CPU 
1051. The KBC 1055 controls an instructing input from an unrepresented 
pointing device or the like. 

The CRTC 1057 controls a display on the CRT 1056. 
2 5 The DKC 1060 controls an access to the HD 1058 and the FD 1059, 

storing a boot program, various applications, an editing file, a user file, a network 
management program, and predetermined processing programs in the present 
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embodiment. The NIC 1052 executes bidirectional data exchange with an 
apparatus or a system on the network 1061. 

(Other embodiments of the present invention) 

The present invention also includes a case of supplying a computer in an 
5 apparatus or a system connected with various aforementioned devices so as to 
operate these devices for realizing the functions of the aforementioned 
embodiments, with program codes of a software realizing the functions of the 
aforementioned embodiments, and operating the various devices according to the 
program stored in the computer (or CPU or MPU) of such system or apparatus. 

10 In such a case, the program codes themselves of the program realize the 

functions of the aforementioned embodiments, and the program codes themselves 
or means for supplying such program codes to the computer, such as a memory 
medium storing the program codes, constitute the present invention. The 
memory medium storing such program codes can be, for example, a flexible disk, 

15 a hard disk, an optical disk, a magnetooptical disk, a CD-ROM, a magnetic tape, a 
non-volatile memory card or a ROM. 

Such program codes are naturally included in embodiments of the present 
invention not only in a case where the computer executes the supplied program 
codes to realize the functions of the aforementioned embodiments but also a case 

2 0 where the program codes realize the functions of the aforementioned embodiments 
in cooperation with an OS (operating system) or another application functioning 
on the computer. 

Furthermore, the present invention includes a case where the supplied 
program codes are once stored in a memory provided in a function expansion 
2 5 board of the computer or a function expansion unit connected to the computer, and 
a CPU or the like provided in such function expansion board or function expansion 
unit executes all the actual processes or a part thereof based on the instruction of 
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the program codes, thereby realizing the function of the aforementioned 
embodiments. 

5 



